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ABSTRACT 


The  Bituminous  Reference  Sample  Program  of  the  AASHTO  Materials  Reference 
Laboratory  (AMRL)  has  been  in  operation  for  more  than  fifteen  years.  It  is 
one  of  four  sample  programs  conducted  as  a cooperative  undertaking  between 
the  National  Bureau  of  Standards  and  the  American  Association  of  State 
Highway  and  Transportation  Officials  (AASHTO).  This  paper  provides  an 
overview  of  accomplishments  of  this  program. 

In  this  paper  information  is  presented  on  the  development  of  standard 
specifications  and  test  methods,  the  AASHTO,  and  the  AMRL.  Details 
concerning  the  operation  of  the  Bituminous  Reference  Sample  Program  are 
reviewed.  The  value  of  the  laboratory  rating  system  and  the  laboratory 
performance  charts  which  have  been  developed  are  discussed.  Conclusions  are 
made  regarding  the  effectiveness  of  the  program  and  the  accuracy  of 
established  test  precision  limits. 
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A REVIEW  OF  THE  BITUMINOUS  REFERENCE  SAMPLE  PROGRAM  OF  THE 


AASHTO  MATERIALS  REFERENCE  LABORATORY 


1.  EVENTS  LEADING  TO  THE  ESTABLISHMENT  OF  THE  PROGRAM 


1.1  Early  Development  of.  Smclf iflatlons-and.  Tests  for  Road  Materials 

In  1911  the  Office  of  Public  Roads  and  Rural  Engineering  (OPRRE)  of  the  U. 

S.  Department  of  Agriculture  published  Bulletin  No.  38  giving  the  first 
description  of  specifications  and  test  methods  for  bituminous  road 
construction  materials.  Bulletin  No.  38  was  later  revised  and  published  as 
Bulletin  No.  314.  In  1916  a survey  of  State  Highway  Departments  by  the 
OPRRE  determined  that  less  than  one  half  of  the  states  had  proper  facilities 
for  testing  highway  construction  materials.  Some  states  did  not  have  any 
facilities  [1].  A majority  of  the  states  indicated  their  willingness  to 
cooperate  with  the  OPRRE  and  tentatively  agreed  to  send  representatives  to 
a conference  on  materials  used  in  highway  work. 

In  1917*  as  a result  of  the  information  gathered  at  the  conference,  the 
OPRRE  published  Bulletin  No.  555  "Standard  Forms  for  Specifications,  Tests, 
Reports,  and  Methods  of  Sampling  Road  Materials."  This  bulletin  covered 
aggregates,  Portland  Cement,  concrete,  paving  brick,  and  bituminous 
materials.  In  1918  the  OPRRE  published  Bulletin  691  "Typical  Specifications 
for  Bituminous  Materials"  and  Bulletin  704  "Typical  Specifications  for  Non- 
Bituminous  Road  Materials". 

1.2  Origin  of  AASHTO  Standard  Specifications  and  Methods^oiU'e^t 

The  American  Association  of  State  Highway  Officials  (AASHO)  was  organized  in 
1914.  After  several  meetings,  the  members  concluded,  in  1919 » that  a need 
existed  for  uniform  standard  methods  of  testing  and  a Committee  on  Tests  and 
Investigations  was  formed.  Representatives  from  the  States  and  the  Bureau 
of  Public  Roads  (BPR),  (the  BPR  became  the  new  title  for  the  OPRRE),  met  in 
1921  to  standardize  the  work  of  the  state  highway  departments  and  the 
Bureau's  laboratories.  This  marked  the  beginning  of  the  AASHO  Committee  on 
Materials.  Bulletin  949  published  by  BPR  in  1921  was  the  first  set  of 
standards  recommended  by  the  Committee  on  Materials. 

In  1924  the  methods  included  in  BPR  Bulletin  1216  which  superceded  Bulletin 
949  were  adopted  as  an  AASHO  standard.  This  standard  remained  in  effect 
until  1931  when  it  was  superseded  by  the  first  edition  of  "Tentative 
Standard  Specifications  for  Highway  Materials  and  Methods  of  Sampling  and 
Testing." 

During  the  sixties  and  early  seventies  many  State  Highway  Departments  became 
Departments  of  Transportation.  This  conversion  was  the  result  of  the  State 
Highway  Departments  taking  on  additional  responsibilities  in  areas  of 
transportation  other  than  highways.  In  recognition  of  this,  AASHO  changed 
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its  name  to  the  American  Association  of  State  Highway  and  Transportation 
Officials  ( AASHTO ) in  1973. 

In  addition  to  the  early  work  by  the  OPRRE  and  the  State  Highway 
Departments,  The  American  Society  for  Testing  and  Materials  (ASTM)  played  an 
important  role  in  the  standardization  of  test  methods  and  specifications. 
ASTM  Committee  D4  on  Road  and  Paving  Materials  was  formed  in  1903.  Many 
AASHTO  representatives  are  members  of  this  ASTM  committee.  For  this  reason, 
AASHTO  generally  adopts  ASTM  standards.  In  cases  where  the  AASHTO  standards 
are  similar  but  not  technically  identical  to  ASTM  methods,  the  differences 
are  noted  in  footnotes  in  the  AASHTO  Book  of  Standards,  Part  1 - 
Specifications  and  Part  II  - Methods  of  Sampling  and  Testing. 

1.3  The  AASHTO  Materials  Reference  Laboratory  (AMRL) 

Based  on  experience  gained  from  a long  involvement  with  the  activities  of 
the  ASTM  Cement  and  Concrete  Reference  Laboratory,  the  American  Association 
of  State  Highway  Officials  proposed  in  1961  that  a Materials  Reference 
Laboratory,  sponsored  by  the  Committee  on  Materials,  be  established  for 
standardizing  and  improving  the  precision  and  reliability  of  test  methods. 
The  AMRL  began  activities  at  the  National  Bureau  of  Standards  in  October 
1965.  Its  primary  responsibilities  were  and  still  are  to  promote  uniformity 
in  the  testing  of  construction  materials  and  to  improve  the  quality  of  test 
results. 

The  four  principal  functions  of  the  AMRL  are:  (1)  Inspection  of  materials 

testing  laboratories  subscribing  to  the  service,  (2)  Distribution  of 
proficiency  test  samples  to  subscribers,  (3)  Study  of  testing  problems,  and 
(4)  Participation  in  the  work  of  technical  committees.  An  overview  of  the 
AMRL  is  presented  in  a paper  by  Steele  [2]. 

The  remainder  of  this  report  is  limited  to  a review  of  the  second  function 
mentioned  above,  concentrating  on  the  Bituminous  Reference  Sample  Program, 
one  of  four  reference  sample  programs.  It  provides  examples  of  the 
activities  and  benefits  associated  with  all  the  programs. 


2.  THE  BITUMINOUS  REFERENCE  SAMPLE  PROGRAM 


2.1  Frequency  of  Sample  Distribution 

The  Bituminous  Reference  Sample  Program  includes  both  asphalt  cements  (AC) 
and  cutback  asphalts  (CB).  Since  1970,  a matched  pair  of  asphalt  cement 
samples  has  been  distributed  to  participants  twice  a year  (in  March  and 
September).  Prior  to  1970,  samples  were  not  distributed  at  regular 
intervals.  Since  1 973 » a pair  of  cutback  asphalt  samples  has  been 
distributed  in  June  of  each  year.  Prior  to  1973,  four  pairs  of  cutback 
asphalts  were  sent  out  at  approximately  six  month  intervals  over  a one  and 
one  half  year  period  starting  in  August  1967  and  ending  in  March  1969. 
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2.2  Grades  of  Bituminous  Materials  Analyzed 


In  the  testing  program,  the  same  grade  of  asphalt  was  not  distributed  at 
every  interval.  Instead,  a variety  of  grades  were  used  (see  Table  1) 
including  penetration  graded  asphalt,  original  viscosity  graded  asphalt  and 
residue  viscosity  graded  asphalt.  For  example,  penetration  graded  asphalt 
cements  have  included  60/70,  85/100,  120/150  and  200/300  grades.  Asphalt 
cements  graded  on  the  original  viscosity  included  AC- 10  and  AC-20  grades. 
Asphalt  cements  graded  on  the  viscosity  of  the  residue  from  the  rolling  thin 
film  oven  test  included  four  samples  of  the  AR-2Q00  grade.  Cutback  asphalts 
included  MC25Q,  MC70,  MC30,  and  RC250  grades.  Details  concerning  each  pair 
of  samples  distributed  are  shown  in  Table  1 . 

2.3  P^r.ti9i.p^.t.ipji.  in.  .tha.  Prs&ram 

The  number  of  laboratories  receiving  asphalt  cements  has  increased  from  53 
at  the  beginning  of  the  program  to  132  laboratories.  The  number  of 
laboratories  receiving  cutback  asphalts  has  increased  from  63  to  127. 

The  types  of  laboratories  participating  in  the  program  include  State  Highway 
and  Federal  test  laboratories,  producer  and  commercial  test  facilities,  and 
municipal,  university,  and  national  testing  agencies.  Most  of  the 
participants  are  located  in  the  United  States  or  Canada.  However,  some 
registrants  are  located  in  Central  and  South  America  and  South  Africa. 

2.4  Preparation  of  Samples 

The  bituminous  reference  sample  program  was  the  first  program  selected  for 
implementation  by  the  AMRL  because  of  the  relatively  small  amount  of  effort 
required  to  prepare  test  samples.  The  bulk  material  for  the  first  pair  of 
samples  was  obtained  in  five-gallon  (20  L)  containers.  The  asphalt  was 
processed  and  test  specimens  prepared  and  packaged  at  the  Federal  Highway 
Administration  (FHWA)  Fairbanks  Highway  Research  Station  located  in  Langley, 
Virginia. 

All  subsequent  pairs  of  reference  samples  were  prepared  by  AMRL  personnel  at 
the  National  Bureau  of  Standards  according  to  the  guidelines  recommended  by 
the  FHWA.  As  the  participation  in  the  program  increased,  two  55-gallon  (220 
L)  drums  were  used  to  acquire  bulk  material  (one  drum  for  each  sample  of  the 
pair).  Preparation  of  a typical  pair  of  test  samples  involves  heating  and 
blending  the  asphalt  material  in  55-gallon  (220  L)  drums  and  quickly 
dispensing  it  into  one-quart  (1  L)  friction  top  cans  (screw  top  cans  are 
used  for  cutbacks).  Two  one-quart  (1  L)  cans,  one  drawn  from  each  of  the 
two  55-gallon  (220  L)  drums,  are  then  packaged  together  in  a parcel  weighing 
about  three  kilograms.  These  quart  (1  L)  cans  are  then  distributed  to 
participants. 

2.5  Seating  Program  for  Participating  Laboratories 

Pairs  of  reference  samples  are  tested  by  the  latest  AASHTO  or  ASTM  Test 
Methods.  A data  form  listing  the  tests  to  be  made  and  special  instructions 
for  testing  and  reporting  the  data  accompany  each  pair  of  reference  samples. 
The  following  are  included  in  the  program: 
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Bituminous  Asphalt  Cement 


Penetration  of  Bituminous  Materials,  25 °C 
Flash  Point  by  Cleveland  Open  Cup,  (°C) 

Solubility  of  Bituminous  Materials  in  Organic  Solvents 
Specific  Gravity  of  Bituminous  Materials 
Kinematic  Viscosity  of  Asphalts,  135 °C 
Viscosity  of  Asphalts,  T202,  60°C 

Effect  of  Heat  and  Air  on  Asphalt  Materials  (Thin  Film  Oven  Test) 

Effect  of  Heat  and  Air  on  a Moving  Film  of  Asphalt  (Rolling 
Thin  Film  Oven  Test) 

Penetration  of  the  Residue,  25°C 
Kinematic  Viscosity  of  the  Residue,  135 °C 
Viscosity  of  the  Residue,  60°C 

gu.t.bftsk  AffPtol.t 

Kinematic  Viscosity  of  Asphalts,  60°C 
Flash  Point  with  Tag  Open-Cup  Apparatus,  (°C) 

Distillation  of  Cutback  Asphaltic  (Bituminous)  Products,  T78 
Penetration  of  the  Residue,  25 °C 
Kinematic  Viscosity  of  the  Residue,  135°C 
Viscosity  of  the  Residue,  60°C 

2.6  Techniques  Used  to  Analyze  Test  Data 

Analysis  of  test  data  is  based  on  the  paired-sample  system  described  in 
papers  by  Youden  [3]  and  Crandall  and  Blaine  [4].  The  analysis  of  test  data 
is  performed  using  a computer  program  developed  by  H.  T.  Arni. 

2.7  Storage  and  Preliminary.  Analysis  of  Data 

To  prepare  the  data  for  analysis,  data  for  each  laboratory  is  entered  into 
a mass  storage  file  and  coded  with  laboratory  numbers,  sample  pair  numbers, 
test  numbers  and  other  pertinent  information  such  as  the  standard  test 
method  used.  This  data  is  then  sorted  and  a statistical  analysis  for  each 
test  property  made. 

The  first  step  in  the  analysis  is  to  remove  from  consideration  any  test 
values  which  can  be  justifiably  considered  invalid.  In  removing  outliers  a 
trial  computer  run  is  executed  which  repeatedly  eliminates  data  beyond  three 
standard  deviations.  The  test  data  to  be  eliminated  are  selected  based  on 
the  results  of  this  trial  run,  the  shape  of  the  normal  curve,  specific 
guidelines  for  the  removal  of  outliers,  and  the  statistics  from  previous 
pairs  of  samples.  Once  the  outliers  have  been  selected  the  data  are 
reanalyzed  to  develop  valid  data  for  averages,  standard  deviations  and 
coefficients  of  variation. 

2.8  The  Preliminary  Report 

A preliminary  report  is  prepared  and  distributed  to  each  participating 
laboratory  approximately  six  weeks  after  test  samples  are  mailed.  The 
report  shows  the  number  of  laboratories  reporting  test  data  at  that  time, 
the  average  values,  standard  deviations,  and  coefficients  of  variation  on 
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each  of  the  tests  included  in  the  testing  program.  The  purpose  of  this 
report  is  to  provide  the  participating  laboratories  with  information  for 
identifying  abnormal  test  results. 

2.9  The  Final Jtepgrt 

As  soon  as  possible  after  all  of  the  test  data  are  received  and  a 
statistical  analysis  is  completed,  an  individualized  final  report  is 
prepared  giving:  (1)  the  results  of  tests  and  ratings  for  individual 

laboratories,  (2)  a statistical  summary  of  results,  and  (3)  a scatter 
diagram  for  each  test  property  analyzed. 

The  results  of  tests  by  an  individual  laboratory  are  compared  with  the 
averages.  Ratings  are  then  assigned  to  indicate  how  far  the  laboratory’s 
data  deviate  from  these  averages.  The  laboratory  ratings  are  based  on  the 
following  scale: 

Rating  +5,  within  1.0  standard  deviations  of  the  mean; 

Rating  +4,  within  1.5  standard  deviations  of  the  mean; 

Rating  ±3,  within  2.0  standard  deviations  of  the  mean; 

Rating  +2,  within  2.5  standard  deviations  of  the  mean; 

Rating  +1,  more  than  2.5  standard  deviations  of  the  mean; 

Rating  0,  data  excluded  due  to  a large  deviation. 

A summation  of  the  ratings  obtained  by  a given  laboratory  over  a period  of 
time  gives  an  indication  of  that  laboratory’s  proficiency.  Significance 
need  not  necessarily  be  attached  to  a single  low  rating  or  pair  of  ratings. 

A continuing  tendency  to  get  low  ratings  on  several  pairs  of  samples, 
however,  indicates  that  a laboratory  should  reexamine  its  testing  apparatus 
and  testing  procedure. 

2.10  Scatter  Diagrams 

To  show  how  individual  laboratory  data  compares  with  that  of  other 
participants,  scatter  diagrams  are  prepared  for  each  test  similar  to  the  one 
shown  in  Figure  1.  For  each  pair  of  samples  from  each  laboratory,  the 
odd-numbered  test  value  is  plotted  on  the  "x"  (horizontal)  coordinate  and 
the  even-numbered  test  value  is  plotted  on  the  "y"  (vertical)  coordinate. 

The  horizontal  and  vertical  lines  represent  the  median  values.  The  distance 
of  the  point  from  the  center  gives  an  indication  of  deviation  from  this 
median.  A continued  excessive  deviation  indicates  some  corrective  action  is 
necessary. 

2.11  Laboratory.  Performance  Charts 

If  evidence  is  provided  that  a substantial  deviation  in  test  results  exists 
within  a laboratory,  that  laboratory  should  check  the  testing  apparatus  and 
the  procedures  used  by  the  operator.  The  best  way  of  demonstrating  how  well 
the  program  evaluates  a laboratory's  testing  capabilities  is  to  study  the 
performance  charts  shown  in  Figures  2 and  3.  The  horizontal  axis  labeled 
"PAIR  NO."  shows  a scale  of  numbers  from  1 to  10.  These  numbers  refer  to 
the  sequence  of  sample  pairs  distributed  at  six'  month  intervals.  The 
vertical  axis  labeled  "NORMAL  DEVIATE"  gives  a scale  of  numbers  from  -3.0 
to  +3.0. 
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Each  normal  deviate  represents  the  number  of  standard  deviations  the 
laboratory’s  reported  data  is  from  the  average.  For  example,  a normal 
deviate  of  -1.7  is  equivalent  to  a rating  of  -3.  A limit  of  +3.0  standard 
deviations  from  the  average  has  been  imposed  on  all  data.  Any  data  at  or 
beyond  these  limits  is  plotted  +3.0  or  -3.0  on  the  graph.  In  figure  2, 
octagons  represent  the  first,  or  odd-numbered  sample,  and  triangles 
represent  the  second,  or  even-numbered  sample.  A graphic  representation  of 
within-laboratory  precision  is  given  in  the  length  of  the  vertical  lines 
connecting  each  of  the  sample  pairs.  The  closer  a pair  of  symbols  for  a 
given  sample  pair,  the  better  the  laboratory's  repeatability.  Conversely, 
the  farther  apart  the  symbols  the  greater  the  laboratory’s  random  error. 

The  lines  connecting  the  midpoints  between  each  pair  of  symbols  show  the 
trend  in  the  laboratory’s  performance  over  the  period  of  time  covered  by  the 
chart.  Midpoints  between  consecutive  pairs  are  connected  with  a solid  line. 
A dashed  line  is  used  to  connect  midpoints  if  data  points  are  missing. 

The  ideal  situation  is  for  a laboratory  to  have  all  symbols  directly  on  the 
center  line,  demonstrating  exceptional  precision  and  testing  accuracy. 

Points  falling  outside  the  range  of  +2.0  normal  deviates  are  usually 
considered  suspect. 

Figure  2 shows  performance  charts  from  six  laboratories  for  the  kinematic 
viscosity  test  of  asphalt  cement.  The  charts  for  all  six  laboratories  are 
based  on  test  data  from  the  same  ten  pairs  of  samples. 

Figure  3 shows  performance  charts  for  six  test  properties  of  asphalt  cement 
from  each  of  two  laboratories.  The  charts  are  based  on  the  test  data  from 
the  same  ten  pairs  of  samples.  Laboratory  ’’A"  shows  good  control  for  all 
six  test  properties  while  laboratory  ’’B”  shows  poor  control. 


3.  DISCUSSION  OF  RESULTS 


3* 1 Causes  of  Variable  Results 

It  is  possible  that  one  or  more  of  the  following  factors  cause  or  contribute 
to  the  poor  control  shown  in  Figure  3: 

•inexperienced  operator 
•deviation  from  standard  method 

•non-conformity  of  apparatus  with  specification  requirements 

•inaccuracy  of  thermometers 

•lack  of  calibration  where  needed 

•improper  handling  of  sample 

•improper  cleaning  of  apparatus 

•incorrect  calculations 
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3.2  Change,  in  the  Coefficient  of  Variation 


To  show  how  test  precision  varied  from  year  to  year,  the  coefficients  of 
variation  for  the  viscosity  and  penetration  of  asphalt  cements  and  thin  film 
residues  were  plotted  chronologically  (Figures  4 and  5). 

Figure  4 shows  the  coefficients  of  variation  for  the  viscosity  at  60 °C  for 
asphalt  cements  and  thin  film  residues.  Only  the  coefficients  of  variation 
for  AC-20  grade  asphalt  cements  were  plotted  to  minimize  possible 
fluctuations  caused  by  differences  in  the  grades  of  asphalt  cements  tested. 
The  coefficients  of  variation  for  the  original  asphalt  cements  range  from 
about  2 to  7 percent  compared  to  a range  of  7 to  16  percent  for  the  thin 
film  residues.  The  higher  values  for  the  thin  film  residues  are  partially 
due  to  compounding  test  procedures  and  errors.  The  graph  shows  generally 
higher  coefficients  during  the  first  four  years  than  during  the  last  six 
years  of  the  program.  This  implies  that  the  precision  of  test  results  from 
participating  laboratories  has  improved  as  a result  of  participating  in  the 
program. 

The  ASTM  multilaboratory  one-sigma  precision  limit  in  percent  (1S£)  for  the 
viscosity  at  60 °C  of  asphalt  cements  and  thin  film  residues  are  also  shown 
in  Figure  4.J/  The  precision  estimate  for  the  original  asphalt  cements, 
plotted  as  3.53  percent,  was  calculated  by  dividing  the  ASTM  multilaboratory 
two-sigma  precision  limit  in  percent  (D2S3&),  given  in  ASTM  D2171  [6],  by 
2.83.  The  precision  estimate  ( 1S% ) for  the  thin  film  residue,  plotted  as 
11.6  percent,  is  given  in  ASTM  D1754  [7]. 

During  the  1972  to  1976  period  the  coefficients  of  variation  of  the  thin 
film  residues  were  all  above  the  ASTM  precision  estimate.  From  1977  to  1982 
the  values  were  all  below  the  ASTM  estimate  of  precision.  The  data 
indicates  that  with  the  improved  testing  capability  of  the  laboratories  the 
ASTM  precision  estimates  of  11.6  percent  (1S$)  and  33.0  percent  (D2S%)  could 
be  lowered  at  least  for  AC-20  graded  asphalt  cements  to  10.0  percent  and 
28.3  percent  respectively.  The  coefficients  of  variation  for  the  original 
asphalt  cements  are  sufficiently  close  to  the  ASTM  precision  to  warrant  no 
change . 

Figure  5 shows  the  coefficients  of  variation  for  the  penetration  at  25 °C  of 
the  original  asphalt  cements  and  the  thin  film  residues  for  all  asphalt 
cement  samples  distributed  between  1966  and  1981.  The  graph  is  broken  into 
three  segments:  (1)  Restricted  Participation,  (2)  Open  Participation  - 

AC-20* s,  and  (3)  Open  Participation  - Mixed  Grades. 

The  first  segment  includes  the  coefficients  of  variation  for  six  pairs  of 
samples.  These  samples,  which  included  a number  of  grades  of  asphalt 
cements,  were  distributed  primarily  to  State  Departments  of  Transportation 
before  the  program  was  open  to  general  participation  (see  Table  1).  The 
coefficients  of  variation  indicate  a general  tendency  toward  increased 


iy  A description  of  the  ASTM  terms  of  precision  applying  to  the  test  methods 
for  construction  materials  is  given  in  ASTM  C670  [5]. 
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precision  over  time  for  both  the  penetration  of  the  original  asphalt  cements 
and  thin  film  residues. 

The  second  segment  of  the  graph  shows  the  coefficients  of  variation  for 
twenty-six  samples  of  asphalt  cement  distributed  to  participants  between 
March  1971  and  March  1977.  At  the  beginning  of  this  period  the  program  was 
opened  to  general  participation.  The  number  of  participants  increased  from 
77  to  122.  Except  for  the  first  four  samples  distributed  (see  Table  1, 
samples  21  to  24)  all  samples  were  either  AC-20  or  85/100  graded  asphalt 
cements.  The  coefficients  of  variation  of  the  original  asphalt  cements  for 
the  twenty- two  similar  samples  were  very  stable  ranging  from  3.15  to  4.27 
percent.  The  gradual  declining  slope  of  the  line  connecting  the 
coefficients  of  variation  for  the  thin  film  residues  indicates  improvement 
in  test  precision  during  this  period. 

The  third  segment  of  the  graph  plots  the  coefficients  of  variation  for  nine 
pairs  of  asphalt  cements.  The  pairs  distributed  during  this  period  varied 
considerably  in  grade  and  source.  As  a result,  the  coefficients  of 
variation  for  both  the  original  asphalt  cements  and  the  thin  film  residues 
are  generally  larger  than  those  at  the  end  of  the  second  segment. 

The  ASTM  multilabcratory  one-sigma  precision  limit  in  percent  (IS?)  for  the 
penetration  at  25° C of  asphalt  cements  above  fifty  penetration  units,  given 
in  ASTM  D5  [8],  is  2.8  percent.  Except  for  six  coefficients  of  variation  in 
the  first  segment  of  the  graph,  all  values  are  larger  than  the  ASTM 
precision  estimate.  In  general,  the  values  are  within  one  unit  of  the 
precision  criteria. 

An  estimate  of  the  test  precision  for  the  penetration  of  thin  film  residues 
is  not  provided  in  ASTM  D1754. 

3.3  g-ontinuous.  _vs  Limited  Participation 

Figure  6 compares  the  coefficients  of  variation  of  the  kinematic  viscosity 
of  asphalt  cements  at  135° C for  continuous  and  limited  participation  groups. 
The  individual  laboratories  comprising  the  continuous  participation  group 
performed  the  kinematic  viscosity  test  on  80  percent  or  more  of  the  asphalt 
cement  samples  since  the  beginning  of  the  program.  The  limited  or 
discontinuous  participation  group  tested  fewer  samples. 

The  plot  of  the  continuous  participation  group  shows  declining  coefficients 
of  variation  during  the  first  five  years  of  the  program  (1966-1971).  After 
1971  the  coefficients  of  variation  varied  between  1 and  3 percent  for  the 
remainder  of  the  program.  Except  for  three  cases  the  coefficients  of 
variation  for  the  continuous  participation  group  were  lower  than  those  for 
the  limited  participation  group,  implying  that  consistant  participation 
improves  test  precision. 

The  ASTM  multilaboratory  one-sigma  precision  limit  in  percent  ( 1 S% ) for  the 
kinematic  viscosity  of  asphalt  cements  at  135° C,  given  in  ASTM  D2170  [9],  is 
3.1  percent.  The  graph  indicates  that  the  test  precision  for  the  continuous 
participation  group  is  usually  better  than  the  ASTM  precision  estimate  while 
the  limited  participation  group,  since  1975,  performed  much  closer  to  the 
ASTM  precision  criteria. 
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3.4  Coefficients  of  Variation  for  Cutback  Asphalts 

Figure  7 shows  the  coefficients  of  variation  obtained  from  the  viscosity 
tests  performed  on  original  cutback  asphalts  and  distillation  residues. 

Three  sets  of  points  sire  plotted,  one  for  the  kinematic  viscosity  of  the 
original  cutback  asphalt,  another  for  the  kinematic  viscosity  of  the 
distillation  residue  at  135° C,  and  a third  for  the  viscosity  of  the 
distillation  residue  at  60 °C.  The  eighteen  cutback  asphalts  analyzed  were 
distributed  to  participants  between  1973  and  1980  as  shown  in  Table  1.  The 
ranges  in  the  coefficients  of  variation  for  the  eighteen  samples  were: 

1)  1.5  to  5 percent  for  the  kinematic  viscosity  at  60°C  on  original. 

2)  6 to  10  percent  for  kinematic  viscosity  at  135°C  on  residue. 

3)  12  to  21  percent  for  vacuum  viscosity  at  60°C  on  residue. 

The  ASTM  multilaboratory  one-sigma  precision  limit  in  percent  (1S$)  for 
liquid  asphalts  at  60°C  having  a viscosity  below  3000  centistokes 
(0.003  M2/S),  given  in  ASTM  D2170,  is  1.06  percent.  This  estimate  appears 
to  be  too  low.  The  ASTM  Test  Method  D402  [10]  gives  no  criteria  for  judging 
the  viscosity  tests  on  distillation  residues,  however,  the  values  plotted  in 
Figure  7 could  serve  as  a means  of  estimating  multilaboratory  precision. 
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4.  BENEFITS  PROVIDED  BY  THE  PROGRAM 


The  Bituminous  Reference  Sample  Program  serves  the  needs  of  the  AMRL 
sponsors  and  the  participating  laboratories. 

The  benefits  include: 

1.  A means  of  checking  both  instrument  and  operator  under  actual 
testing  conditions. 

2.  A means  of  comparing  individual  test  results  with  the  average 
of  a large  body  of  results  so  that  corrective  action  may  be 
taken  where  wide  discrepancies  in  test  results  occur. 

3.  A means  of  evaluating  the  quality  of  the  individual  measurement, 
thereby  reducing  the  risk  of  dispute  due  to  testing  errors. 

4.  A means  of  furnishing  documentation  of  testing  capabilities. 

5.  Improved  reliability  of  asphalt  cement  and  cutback  asphalt 
test  measurements,  and 

6.  Better  definition  of  the  precision  of  standard  methods  of  test. 

Providing  these  services  is  one  way  the  AMRL  meets  its  primary 
responsibility  of  promoting  uniformity  in  the  testing  of  construction 
materials  and  the  quality  of  test  results. 
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5.  CONCLUSIONS 


1.  The  individual  laboratory  ratings  on  a pair  of  reference  samples 
provide  a good  indication  of  the  accuracy  of  laboratory  test 
results. 

2.  The  laboratory  performance  charts  provide  a method  of  demonstrating 
a laboratory’s  testing  capabilities. 

3.  The  coefficients  of  variation  for  the  viscosity  at  60°C  of  AC-20 
graded  asphalt  cements  and  thin  film  residues  indicate  that  the 
precision  of  participating  laboratories  has  improved  since  1972. 

4.  As  illustrated  by  the  test  for  the  kinematic  viscosity  of  asphalt 
cements  at  135°C,  the  precision  of  results  is  generally  better 
for  laboratories  that  participate  continuously  in  the  bituminous 
reference  sample  program  than  for  those  that  participate  infrequently. 

5.  Coefficients  of  variation  for  the  penetration  and  viscosity  of  thin 
film  oven  and  distillation  residues  are  higher  and  more  variable  than 
those  for  the  original  asphalt  cements  and  cutback  asphalts. 

6.  A review  of  the  coefficients  of  variation  obtained  from  tests 
performed  on  asphalt  cement  and  cutback  asphalt  reference  samples 
provides  a useful  means  of  evaluating  established  multilaboratory 
precision  limits  or  determining  test  precision  where  estimates 

do  not  yet  exist. 


I 


11 


6.  REFERENCES 


[1]  Page,  L.  W. , "Standard  Forms  for  Specifications,  Test,  Reports,  and 
Methods  of  Sampling  for  Road  Materials",  U.  S.  Department  of  Agriculture 
Bulletin  No.  555,  Nov.  1917. 

[2]  Steele,  G.  W. , "The  AASHTO  Materials  Reference  Laboratory  - Fifteen  Years 
of  Service",  AASHTO,  Vol.  60,  No.  1,  1981,  p.35. 

[3]  Youden,  W.  J.,  "Statistical  Aspects  of  the  Cement  Testing  Program", 
Proceedings,  American  Society  for  Testing  and  Materials,  Vol.  59,  1959, 

p.  1120. 

[4]  Crandall,  J.  R.  and  Blaine,  R.  L. , "Statistical  Evaluation  of  Interlabora- 
tory Cement  Tests",  Proceedings,  American  Society  for  Testing  and  Materials, 
Vol.  59,  1959,  p.  1129. 

[5]  ASTM  C670,  Standard  Practice  for  Preparing  Precision  Statements  for  Test 
Methods  for  construction  Materials,  Annual  Book  of  Standards,  Parts  13, 

14,  17  and  19, American  Society  for  Testing  and  Materials,  1916  Race 
Street,  Philadelphia,  PA  19103. 

[6]  ASTM  D2171,  Standard  Test  Method  for  Viscosity  of  Asphalts  by  Vacuum 
Capillary  Viscometer,  Annual  Book  of  Standards,  Part  15,  American  Society 
for  Testing  and  Materials,  1916  Race  Street,  Philadelphia,  PA  19103. 

[7]  ASTM  D1754,  Standard  Test  Method  for  Effect  of  Heat  and  Air  on  Asphaltic 
Materials  (Thin  Film  Oven  Test),  Annual  Book  of  Standards,  Part  15, 

American  Society  for  Testing  and  Materials,  1916  Race  Street,  Philadelphia, 
PA  19103. 

[8]  ASTM  D5,  Standard  Test  Method  for  Penetration  of  Bituminous  Materials, 

Annual  Book  of  Standards,  Part  15,  American  Society  for  Testing  and 
Materials,  1916  Race  Street,  Philadelphia,  PA  19103. 

[9]  ASTM  D2170,  Standard  Test  Method  for  Kinematic  Viscosity  of  Asphalts 
(Bitumens),  Annual  Book  of  Standards,  Part  15,  American  Society  for  Testing 
and  Materials,  1916  Race  Street,  Philadelphia,  PA  19103 

[10]  ASTM  D402,  Standard  Test  Method  for  Distillation  of  Cut-Back  Asphaltic 
(Bituminous)  Products,  Annual  Book  of  Standards,  Part  15,  American  Society 
for  Testing  and  Materials,  1916  Race  Street,  Philadelphia,  PA  19103. 


12 


TABLE  1 


BITUMINOUS  REFERENCE  SAMPLE  PROGRAM  PARTICIPATION 
FROM  1966  TO  1982 


Sample 
Nos . 

Mate- 

rial 

Grade 

Date 

Distributed 

State 

Fed- 

eral 

Pro- 

ducer 

Commer- 

cial 

Miscel- 

laneous 

Total 

01 

AC 

85/100 

05/66 

52 

1 

53 

02 

AC 

200/300 

! 03 

AC 

85/100 

01/67 

52 

1 

53 

04 

AC 

60/70 

i 05  & 06 

AC 

85/100 

03/67 

57 

1 

— 

— 

1 

59 

07  & 08 

CB 

MC-250 

08/67 

59 

3 

— 

— 

1 

63 

09  & 10 

CB 

MC-250 

03/68 

59 

3 

— 

— 

1 

63 

11  & 12 

CB 

MC-30 

09/68 

59 

3 

2 

1 

1 

66 

13  & 14 

CB 

RC-250 

03/69 

66 

4 

2 

1 

1 

74 

i 15 

AC 

85/100 

09/69 

66 

7 

2 

1 

1 

77 

16 

AC 

60/70 

1 17  & 18 

AC 

120/250 

03/70 

66 

7 

2 

1 

1 

77 

1 19 

AC 

120/250 

09/70 

66 

7 

2 

1 

1 

77 

20 

AC 

200/300 

21  & 22 

AC 

120/150 

03/71 

66 

9 

15 

30 

2 

122 

23  & 24 

AC 

200/300 

09/71 

66 

9 

15 

29 

2 

121 

25  & 26 

AC 

85/100 

03/72 

66 

9 

29 

21 

3 

128 

27  & 28 

AC 

AC-20 

09/72 

66 

9 

29 

17 

2 

123 

29  & 30 

AC 

AC- 20 

03/73 

66 

11 

26 

17 

1 

121 

, 31  & 32 

CB 

MC-250 

06/73 

65 

8 

19 

18 

3 

113 

33  & 34 

AC 

AC-20 

09/73 

66 

9 

25 

17 

— 

117 

35  & 36 

AC 

AC-20 

03/74 

75 

6 

28 

14 

3 

126 

37  & 38 

CB 

MC-250 

06/74 

75 

6 

21 

8 

2 

112 

39  & 40 

AC 

85/100 

09/74 

76 

6 

23 

12 

2 

119 

41  & 42 

AC 

AC-20 

03/75 

76 

7 

20 

11 

4 

118 

43  & 44 

CB 

MC-250 

06/75 

76 

7 

17 

7 

3 

110 

45  & 46 

AC 

85/100 

09/75 

76 

7 

19 

9 

4 

115 

47  & 48 

AC 

AC-20 

03/76 

79 

7 

22 

10 

4 

122 

49  & 50 

CB 

MC-250 

06/76 

79 

6 

18 

7 

3 

113 

51  & 52 

AC 

AC-20 

09/76 

78 

7 

21 

11 

4 

121 

53  & 54 

AC 

AC-20 

03/77 

78 

7 

22 

8 

4 

119 

55  & 56 

CB 

MC-250 

06/77 

75 

6 

19 

5 

3 

108 

57  & 58 

AC 

AC-10 

09/77 

78 

7 

22 

9 

5 

121 

59  & 60 

AC 

AC- 20 

03/78 

79 

7 

25 

9 

6 

126 

61  & 62 

CB 

MC-250 

06/78 

76 

6 

22 

5 

3 

112 

j 63  & 64 

AC 

AR-20 

09/78 

76 

7 

25 

9 

6 

123 

65  & 66 

AC 

AC-20 

03/79 

72 

9 

26 

11 

5 

123 

67  & 68 

CB 

MC-250 

06/79 

74 

8 

21 

7 

3 

113 

69  & 70 

AC 

AC-10 

09/79 

72 

7 

26 

11 

5 

121 

71  & 72 

AC 

AC-20 

03/80 

72 

7 

27 

13 

6 

125 

73  & 74 

CB 

MC-70 

06/80 

65 

7 

22 

15 

5 

114 

75  & 76 

AC 

AC- 20 

09/80 

65 

7 

28 

18 

8 

126 

1 77  & 78 

AC 

AR-20 

03/81 

65 

8 

32 

17 

10 

132 

79  & 80 

CB 

MC-250 

06/81 

65 

7 

27 

18 

10 

127 

! 81  & 82 

AC 

AC-20 

09/81 

65 

8 

32 

17 

10 

132 

13 
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‘POISES  SAMPLE  83 

FIGURE  I - VOUOEN  SCATTER  DIAGRAM  FOR  VISCOSITY  AT  60C 

*NOTE : The  SI  unit  of  viscosity  is  1 Pa-s {lN*s/m2)  and  is  equivalent  to  10  poises. 
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FIGURE  2 - PERFORMANCE  CHARTS  FOR  SIX  LABORATORIES,  SHOWING  NORMAL  DEVIATES 
FOR  THE  KINEMATIC  VISCOSITY  OF  ASPHALT  CEMENTS  AT  135C 
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FIGURE  3 - PERFORMANCE  CHARTS  FOR  SIX  TEST  PROPERTIES  OBTAINED  FROM  TOO 
DIFFERENT  LABORATORIES.  THE  RESULTS  OF  LABORATORY  A INDICATE  GOOD 
TESTING  CONTROL;  THE  RESULTS  OF  LABORATORY  B INDICATE  POOR  PERFORMANCE. 
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FIGURE  4 - COEFICIENT  OF  VARIATION  FOR  VISCOSITY 
AT  60C  DURING  10  YEARS  OF  LABORATORY  PARTICIPATION 

AC-20  ASPHALT  CEMENTS  AND  THIN  FILM  RESIDUES 
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APPROXIMATE  TEST  PERIOD 

FIGURE  5 - COEFFICIENT  OF  VARIATION  FOR  THE  PENETRATION 
TEST  - ORIGINAL  ASPHALT  CEMENTS  AND  THIN  FILM  RESIDUES 


SOLID  LINE-CONTINUOUS  PARTICIPATION  CROUP 
DASHED  LINE-LIMITED  PARTICIPATION  CROUP 


Z 

O 


CO 

M 

u <*> 

W CD 
OS  -H 

Oi 

s <*> 

E-*  *H 
CO  • 
< ro 


00 

CD 


CD 

1^ 

CD 


r* 

CD 


in 

a 


a 
o 
*— • 

ct 

lli 

CL 


co 

UJ 


UJ 

I- 

< 

x: 


CL 

O 

2- 

2-* 
CL  « 


^zo 


u 


I z 


s ° 

CJ  »-• 


UJ 


in  h- 

K>  < 
•— • 


o 

w K 

a: 

2>-tt 

0. 

oK5 

to 

CL 

< 

(ft  *“•  °- 

[2co 
£ o 

CD 

2w 

E> 
o > 

— 

uu 

• p 

a 

tog 

co 

gs 

CO 

— 1 ^ 

CD 

o* 

UOUJU.lL  OU.  XCL  — <>--«OZ 


19 


M 

LO  — 
M dP 

CJ  to 
W <H 
05  — 
(X 

dP 
2 to 
H O 
to  • 
<C  rH 


a: 

UJ 

CD 

n 

3 

z 

LxJ 

_J 

CL 

z 

< 

co 


UOUJLL  OLl 


O Z X 


20 


FIGURE  7 - COMPARISON  OF  THE  COEFFICIENT  OF  VARIATION 
FOR  VISCOSITY  TESTS  ON  THE  ORIGINAL  AND  DISTILLATION 
RESIDUES  FOR  CUTBACK  ASPHALTS 
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of  test,  the  AASHTO  and  the  AMRL,  is  presented  to  give  a historical  perspective 
and  to  identify  the  need  for  the  program.  Details  concerning  its  operation  are 
reviewed.  The  coefficients  of  variation  derived  from  the  analysis  of  laboratory 
test  data  from  a number  of  selected  tests  are  plotted  and  evaluated.  The  value 
of  the  laboratory  rating  system  and  the  laboratory  performance  charts  which  have 
been  developed  is  discussed.  Conclusions  are  made  regarding  the  effectiveness 
of  the  program  and  the  accuracy  of  established  test  precision  limits. 
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